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..ABSTRACT 

The effects of two cover letter manipulations and 
their interactions with demographic variables on response to the 

a survey were investigated. The two manipulations, 
type of respondent group identification, were 
respondents' perceptions of their social 
position. In all, 599 replies were received from 
and female (72 percent) teachers. Seventy-three 



initial mailing of 
type of appeal and 
intended to affect 
responsibility and 
male (28 percent) 

percent were Anglo, with the remainder largely Hispanic. The appeal 
manipulation was appeal to the respondent's professional expertise or 
appeal to affiliation with a sample of teachers* No simple effects of 
either cover letter was found, although an interactive effect was 
found. Effects of gender and ethnicity on response rate were 
significant, with higher response rates for females and Anglos. Three 
tables present study data. An appendix lists the appeals made. 
(SLD) 
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ABSTRACT 



The purpose of this study was to investigate the effects of 
two cover letter manipulations and their interactions with 
demographic variables on responses to the initial mailing of a 
survey. The two manipulations, type of appeal and type of 
respondent group identification, were intended to affect 
respondents' perceptions of their social responsibility and 
position. No simple effects of either cover letter manipulation 
were found, though an interactive effect was found. Effects of 
gender and ethnicity on response rate were significant, with 
higher response rates for females and for Anglos. 
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Response rate to a mail survey is crucial, and population 
generalization cannot be made with confidence if response rates 
are low. Even if the sample was originally randpm, low response 
rates may compromise the representativeness of the data obtained 
since the respondent group may differ systematically from 
nonrespondents. Thus much effort has been devoted to maximizing 
response rates. Although repeated follow-ups have been clearly 
shown to increase response rates (e.g., Heberlein & Baumgartner, 
1978), from a purely economic perspective, maximizing response to 
the initial mailing is of particular interest since both time and 
follow-up costs are reduced if people respond quickly. 
Unfortunately, investigation of the effects of social 
psychological variables on responses to an initial mailing is 
limited. Subsequently, it is not clear what factors contribute 
to maximizing responses. The purpose of this study was to 
investigate the effects of two survey cover letter manipulations 
and their interactions with demographic variables on response to 
the initial mailing of a survey. The two manipulations, type of 
appeal and type of respondent group identification, were intended 
to affect respondents' perceptions of their social responsibility 
and position. 

The effects of varying cover letter appeals have been 
previously investigated, albeit not extensively, with 
contradictory results. The appeals used have emphasized the 
social utility of responses (utility) , benefits to the sponsor 
(sponsor) , or benefits to the respondent (user) . A social 
utility appeal emphasizes the importance of the survey in the 
betterment of the general population versus its importance in 
aiding either the sender (sponsor) or the respondent (user) . 
Social utility appeals have included phrases such as "contribute 
to understanding more about," "a scientific examination of 
how. . . " Sponsor appeals have been phrases as "Your opinions are 
very important to our successful completion of this study," "help 

us understand " User appeals have been phrased as "It's 

important for you to express your opinion so that — ", "you will 
be better served as a result of ..." Respondents in these 
studies have been from groups as diverse as nurses, presidents of 
Fortune 500 companies, life insurance salespeople, city planners, 
businessmen, university faculty, engineers, university alumni, 
and the general population. Study results have been mixed. 
Studies have found no significant effect of type of appeal 
(Childers, Pride, & Ferrell, 1980—2-3% difference; Lmsky, 1965- 
-.9% difference), a positive effect of social utility over 
sponsor appeal (Jones & Linda, 1978—4.7%); a negative effect of 
social utility versus sponsor and user appeals (Childers et al., 
1980—10-11%; Tyagi, 1989—21.4%); a positive effect of user over 
sponsor appeal (Champion & Sear, 1969—3.6%; Jones & Linda, 1978- 
-4.7%); a negative effect of user over sponsor appeal (Kerin & 
Harvey, 1976) ; an interactive effect of appeal and sponsoring 
organization with higher returns for a user appeal from a 



commercial organization and higher returns for a sponsor appeal 
from a university (Houston & Nevin, 1977); an interaction of 
appea? l^l with'^aAonymity and feedback (Tyagi, 1989 ; and no 

interactive effects of appeal, ^P°'^^°^ ' P° J55??.^"ne?sus 

Nevin 1977) or of appeal and presentation (handwritten versus 
tZll) (Childers et al., 1980). The strongest effects of appeal 
wSe foind in the Childers et al. (1980) and Tyagi (1989) 
studies, in the Childers et al. study, appeal effects were found 
for a sample of 700 business academicians but no effects of tne 
same treatments for a sample of 1,000 business practitioners. 

Sletto {1940) used a somewhat different cover letter 
manipulation Three different letters that emphasized helping 
othe?s, helping education as an enterprise, and challenging the 
recipient to do something that "can't ' fSiahest 

university alumni. The altruistic appeal yielded the highest 
Response rate (67%) compared to 64 and 60% for the other two 
types of letters but differences were not significant. Tne 
author argues this lack of significance was due to the small 
sample size. Hendricks et al. (1972) manipulated enhancement of 
the respondent and/or of the sponsor, which they termed 
ingratiation. They found the only significant effect to be a 
depressed return rate for the interaction of respondent and 
sponsor ingratiation in a high effort condition (7-page 
qSestionnaire vs. 1-page). They found that no flattery or too 
much flattery (respondent plus sponsor ingratiation) lowered 
returns when time required for survey completion was higher. The 
authors suggested the existence of an implicit norm for polite 

^^^^'^jon;s and Linda (1978) suggested that differences in results 
across studies may be due to the use of nonstandard appeals and 
also to population differences. Salience of the survey may also 

^^^^ HebSrl^in and Baumgartner pointed out in a 1978 review paper 
that mail survey research is notably atheoretical . This 
statement applies to the research on the effects of appeals. 
But two potential theoretical bases exist for research on 
app4als? ^One is in the development of the ^^l^^ionship between 
attitude and behavior. Responding to a survey is a behavior, and 

as such, may be predicted just as other ^jlj^J^^JI^^f ;ither 
Fishbein (1980) suggested that variables that affect either 
attitude or the expectations the person Perceives others to hold 
about the behavior affect the behavior. McKillip (1984) stated 
that "Techniques that identify the recipient personally 
terns o? somfgroup will make the expectations ^^d norms of the 
culture or of the particular reference group more salient (p. 
8?) ?hus, framing an appeal so that it reinforces a culturally- 
approved norm should affect response rate. 

A second theoretical base is in the construct °- ;90iJ^. 
specifically pseudoaltruism. Egoism is defined as a ^°tivatiou.l 
sta?e with the ultimate goal of increasing one's own welfare 
fBatson 1991) . Pseudoaltruism is a special form of egoistic 
mo?iva?ion in which rewards are internal rather than external 



(e.g., reduced guilt, enhanced self-concept). Pseudoaltruistic 
behavior is motivated by personal norms, situation-specific 
expectations stemming from internalized values, with self- 
administered rewards and sanctions. What is termed 
pseudoaltruism by Batson (1991) was called altruism by Schwartz 
and his associates (1977, 1982). Staub (1978) described this 
pseudoaltruistic behavior as prosocial acts that appear to be 
intended to benefit others but which are associated with internal 
rewards if not with material or social gain. Aronfreed (1970) 
argued that action for internal gain should not be viewed as a 
form of social exchange, stating that the concept of altriusm is 
more useful in this circumstance. Response to a mail survey, in 
the absence of material incentives, could be viewed as altruistic 
or pseudoaltruistic behavior. Social utility and sponsor appeals 
are direct requests for such pseudoaltruistic behavior; user 
appeals are more closely tied to personal material or social 
gain. The literature suggests the possibility that 
characteristics of the sample may affect the likelihood of such 
responses. Eisenberg et al. (1989) found several interesting 
predictors of a measure of helping. For women, ascription of 
responsibility correlated with helping (r = .40) along with 
empathy (r = .38). For men, social responsibility correlated 
with helping (r = .35) as did empathic concern (r = .37). 
Results of this and other studies suggest the possibility that 
gender or other personological variables, be they inherited or 
societally prescribed, correlate with pseudoaltruistic behavior. 
Other researchers (e.g., Schwartz, 1977) have argued that the 
impact of personality depends upon the situational context and 
the behavior required; that personally held norms influencing 
response may not be consistently related to any personality 
measures. However, Eisenberg et al.'s (1989) results suggest 
that variables such as gender may affect behavior, in this case 
initial response to the survey. 

This study sought to assess the effects on response rate of 
user and social utility appeals in combination with emphasis on 
either the respondents' professional expertise or affiliation 
with a sample of teachers (2x2 design) . The appeal manipulation 
is similar to that used in previous research, but applied with a 
group of teachers. The manipulation of group identification or 
emphasis was intended to create an authority or affiliation 
perspective on the part of respondents. It was hypothesized that 
a user appeal would yield more responses for this group of 
teachers than a social utility appeal and that emphasis on 
authority versus affiliation would interact with gender. Males 
would stereotypicall^ be thought to respond more positively to an 
appeal to authority and females to an appeal for affiliation or 
support. Also examined were the interactions of these 
manipulations with demographic characteristics in addition to 
gender (ethnicity, community size, and grade level taught) . The 
appeals used are presented in Appendix A. 
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METHOD 

Subjects , 

The 1500 survey participants were randomly selected from a 
State Department of Education file containing names and addresses 
of all licensed teachers in public schools in New Mexico during 
the fall of 1991, Only personnel who were in teaching positions 
(regular education and special education) were included in the 
sample. Responses were received from 607 people (40,7%). Five 
respondents removed the code number from their questionnaire and 
so could not be used in this study; an additional three persons 
had incorrectly entered code numbers and these cases could not be 
matched with the demographic data file leaving a sample of 1492 
cases and 599 respondents. 

Hales accounted for 28% of the total sample and females 72%. 
The proportion of male teachers statewide in 1991-92 was 26.9% 
and females 73.1%. Anglos accounted for 73% of the sample with 
24% Hispanic, and 3% African-American, Asian, or American 
Indian. The proportion of Anglo teachers statewide for 1991-92 
was 72.7% with 24.1% Hispanic. 

Elementary level teachers comprised 53% of the sample with 
15% middle school, 6% junior high, and 24% high school. The four 
cases who reported they were special education teachers were^ 
dropped from the analysis for level taught (n=1488) . Community 
size was determined from U.S. Bureau of the Census information 
and was based on school zip codes. Small town was the most 
frequent school location (31%) with 29% from mid-sized cities, 
17% from rural areas, 13% from large towns, and 10% from fringe 
areas for mid-sized cities. 

Instruments . The survey addressed teachers' attitudes 
toward assessment. It contained 47 multi-part questions 
(approximately 105 responses, depending upon content areas 
taught) presented on 4 two-sided pages or 8 one-sided pages. 
Items were primarily closed-response with only two questions 
requiring brief written- in responses. A cover letter was 
included with each survey. Four versions of the cover letter 
that corresponded to the four appeals were written and one was 
included with each survey. 

Procedure. Based on random number assignment, participants 
were sent one of four appeal letters with their surveys. Surveys 
were mailed to teachers from the State Department of Education 
April 22-23, 1992. Responses were accepted until July 6, 1992. 
There were no follow-up mailings. 

Data were analyzed using a logit model (Kennedy, 1992) to 
assess the effects of appeal (utility and user) , emphasis 
(authority and affiliation), gender, ethnicity (Anglo and 
Hispanic), community size (5 levels), and grade level taught 
(elementary, middle, high school) on responses. Two logit 
analyses were conducted, however, since inclusion of all 
variables simultaneously resulted in numerous cells with no 
cases. The first analysis included the variables appeal, 
emphasis, ethnicity, and gender. Only Anglos and Hispanics were 
included; there were too few African-Americans, American Indians, 



or Asian- Americans to provide a meaningful analysis for these 
groups. The second analysis included appeal, emphasis, community 
size, and grade level taught. Likelihood ratio chi-square values 
were used as the test statistics and Akaike's Information 
Criterion (AIC) was calculated for model comparisons (Agresti, 
1990). Yule's Q was calculated to express association values tor 
categorical variables (2x2 tables only). Yule's Q may be 
interpreted in the same manner as a Pearson correlation 
coefficient. 

RESULTS 

In the first analysis, the null logit model did not fit the 
data (l2 = 32.92, d.f. = 15, p<.005). The most parsimonious 
fitting model included all terms in the null logit model (all 
main marginals and all fixed-term interactions) plus the effects 
of gender and ethnicity on response {1/ = 9.97, d.f. - 13, 
p>.69). While significant effects of gender and ethnicity on 
response were found, there were no significant effects of aPP^al. 
The appeal by emphasis by ethnicity effect was significant at the 
.10 level (partial L^ = 2.92, p<.09) although a model without 
this term demonstrated adequate fit. When this interaction term 
was added to the model, fit improved (L*^ = 2.67, d.f. - 7, 
p> 91) . The AIC was calculated for the simpler and more complex 
fitting models. The AIC value ^as lower for the model including 
the complex interaction term, indicating that the more complex 
model provides a better explanation of the data. Table 1 
presents the frequencies and proportions of respondents and 
nonrespondents for appeal, emphasis, gender, and ethnicity. 
Females responded at a 7.8% higher rate than males. Table 1 
includes an "other" category for ethnicity even though these 
cases were not included in the logit analysis. Anglos had the 
highest response rates (12.1% higher than Hispanics) . Table 2 
presents partial and marginal values for all terms in the 
model. Table 3 presents similar information for the interaction 
term. An authority appeal was slightly more effective than an 
affiliation appeal for Anglos under both user and socral utility 
conditions; an authority appeal was more effective for Hispanics 
under a social utility appeal but 1 ;.ss effective with a user 
appeal. Yule's Q values are clearly different for the Anglo and 
Hispanic samples. . ^ , . , , 

In the second analysis, the null logit model provided 
adequate fit to the data (L^ = 46.19, d.f. = 59, p>.88). Thus no 
significant effects of community size, level taught, appeal, or 
omphasis on response were found and no interactive effects ^ 
(although the community size term had a partial chi-square--L - 
9.46, p<.06~significant at the .10 level). In this analysis, 
cell sizes were a concern since of 120 cells, 29 had cell 
frequencies less than 5. The community size categories of 
sized city and fringe for mid-sized city were collapsed and the 
analysis rerun. Again, the null logit model provided good fit to 
the data. Table 2 presents respondent and nonrespondent 
information by the original community size variable. The lowest 
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response rates were from schools in icid-c:ized cities and rural 
areas. 

DISCUSSION 

The nature of r.he appeal produced no significant main effect 
on response rates in this study for this sample of teachers. 
This result agrees with that Childers et al. (1980) and Linsky 
(1965) found for samples of business practitioners and nurses. 
There also was no significant main effect of emphasis, suggesting 
that authoritative versus affiliative appeals made little 
difference overall. There was, however, an interaction term 
suggesting that appeal and emphasis had a differential effect for 
Anglos and Hispanics, authority working best for Anglos under 
both user and social utility conditions while affiliation worked 
better for Hispanics under the user appeal. This result suggests 
that effects of appeal and emphasis, if these manipulations do 
indeed actuate cultural norms, are dependent upon the culture of 
the audience. This finding is supported by the result of 
significant differences in response rates for different ethnic 
groups (about 1/3 for Hispanics but close to 1/2 for Anglos) . 
Diverse tactics may be required to encourage responses from 
culturally diverse groups. If surveys are to capture a large 
proportion of the audience in non-Anglo groups, special attention 
needs to be given to understanding the general culture of that 
audience and in designing instruments that encourage response 
from groups who may use the English language somewhat differently 
than Anglos and who may have differing beliefs. In accord with 
McKillip (1984), framing appeals to reinforce culturally-approved 
and culturally-salient norms may produce the most positive 

Females responded at a higher rate than males. This result 
is in contrast to that of Green and Kvidahl (1989) who found no 
gender effect and that of Green and Stager (1986) who found males 
to respond at a higher rate to a first mailing. The New Mexico 
sample was more ethnically diverse than the Green and Stager 
sample but both were drawn from Western, primarily rural-small 

town areas. . , _ 

The effects that were found were small (Yule's Q values of 
.16 to .24). Two explanations of the small size of the effects 
are, first, that survey recipients may not have even read the 
cover letter and, second, that the cover letter wording was not 
sufficiently strong. Effects found in previous research have 
generally been small as well. 

It is suggested that careful attention be paid to the cover 
letter but that further investigation is needed to ascertain the 
most appropriate appeal for use with different populations. For 
example, subsequent research could extend the present study to 
include different survey sponsors, with more explicit tailoring 
of the appeal for different ethnic, and possibly socio-economic 
groups. An extension of this study to populations of 
businesspersons and general consumers is also needed. Effects ot 
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social utility and user appeals may differ if a nonacademic group 
is surveyed by a nonacademic organization. Consistency of the 
nature of the sponsoring organization, the type of appeal, and 
audience may be more important In encouraging responses than 
any one factor alone. That is, a social utility appeal may work 
better if an academic/nonprofit organization sponsors a survey of 
an academic sample while a user appeal may be more effective with 
a consvimer group survey sponsored by a for-profit company. The 
notion of appeals for pseudoaltruistic behavior may be of 
secondary importance in contrast to establishing consistency of 
method and credibility of the sponsoring organization in 
conducting that survey. 
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Table 1 



Frequencies and Proportions of Respondents and N onrespondents by 
A ppeal. Emphasis. Gender. Ethnicity, and Location 

Responded Failed to Respond 
Variable n | D 1 

Appeal: User 346 46.1 404 53.9 

Social Utility 353 47.1 397 52.9 

LR Chi-square = .13, p>.71; Q = .02 

Emphasis: Support 341 45.5 409 54.5 

Authority 358 47.7 392 57.3 

LR Chi-square = .77, p>.37; Q = .05 

Gender: Male 174 41.0 251 59.0 

Female 525 48.8 550 51.2 

LR Chi-square = 7.67, p<.01; Q = . 16 

Ethnicity: Anglo 546 49.9 549 50.1 

Hispanic 135 37.8 222 62.2 

(Other 18 37.5 30 62.5) 



LR Chi-square 


= 17. 


52, p<.001; 


Q = • 


24 


Mid-sized city 


182 


41.4 


258 


58.6 


Fringe — mid- 


73 


50.0 


73 


50.0 


sized city 










Large town 


93 


49.2 


96 


50.8 


Small town 


237 


50.5 


232 


49.5 


Rural 


114 


44.5 


142 


55.5 



LR Chi-square = 9.42, p<.06 
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Table 2 

Partial and Marginal Associations for Loait Model Effects 



Partial Marginal 
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.55 
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. 25 
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.32 


.25 
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.31 


.07 


4 


.27 


.04 




7 


.55 


. 01 


8 


.63 


.01 


Ethnicity x Response 


14 


.27 


.01 


15 


.42 


.01 


ADDeal X Emphasis x Gender 


2 


.09 


. 15 
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.16 


.14 


AiDneal X EmDhasis x Ethnic itv 




.42 


.52 
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.46 


Appeal X Gender x Ethnicity 
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.61 
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.06 


ETnr)ha<5is x Gender x Ethnicitv 
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.61 
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Appeal X Emphasis x Response 
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Emnhasis x Gender x Response 
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. 18 


Emphasis x Ethnicity x Response 
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Note . All effects have 1 degree of freedom. 



Table 3 

Appeal bv Emphasis by Ethnicity Effe ct: on Response Rate 



Anglo Hispanic 

Response Nonresponse Response Nonresponse 

Condition n % n 1 Q 1 D 3l_ 

Us6ir * 

Authority 144 50.9 139 49.1 28 35.0 52 65.0 

Affiliation 125 46.0 147 54.0 40 44.4 50 5->.5 

Q = .10 Q = --20 

Social Utility: ^ .^^ 

Authority 141 52.6 127 47.4 38 40.4 56 59.6 

Affiliation 136 50.0 136 50.0 29 31.2 64 68.8 

Q = .05 Q = +-20 



Appendix A 



User Appeal. Authority 



As a professional in education, we ask that you contribute your 
expertise to an appraisal of assessment proficiencies and 
directions for New Mexico... To this end, a questionnaire has been 
designed to help us gather information about your interests and 
perceptions. You can expect to be better understood and served 
as a result of this study Your ideas are valuable to us. 

User Appeal. Affiliation 

AS a fellow educator, we ask that you join with other New Mexico 
teachers in an appraisal of assessment proficiencies and 
directions for New Mexico... To this end, a questionnaire has been 
designed to help us gather information about your interests and 
opinions. You can expect to be better understood and served as a 
result of this study Your opinions are valuable to us. 

Social Utility A p peal. A uthority 

As a professional in education, we ask that you contribute your 
expertise to an appraisal of assessment proficiencies and 
directions for New Mexico... To date, there has not been a 
scientific examination of current teacher assessment practices 
and opinions about those practices. We ask that you contribute 
to this important area of inquiry. Your responses are important 
in providing an accurate picture of current and ideal 
practice. . .Your ideas are valuable to us. 

finr-.ial Utili ty A p peal. Af filiation 

AS a fellow educator, we ask that you join with other New Mexico 
teachers in an appraisal of assessment proficiencies and 
directions for New Mexico. . .To date, there has not been a 
scientific examination of current teacher assessment practices 
and opinions about those practices. We ask that you <=°ntribute 
to this important area of inquiry .. .Your opinions are valuable to 

US. 



